Energy-efficient neuromorphic nonlinear adaptive control

Abstract

We present a neurally inspired control system that adapts to unknown dynamics and unknown kinematics. This control system is an adaptation of Cheah, Liu, and Slotine (2005) adaptive tracking control, adjusted to allow for extremely power-efficient implementation using analog neuromorphic hardware. We demonstrate the performance of the algorithm in simulation and in a physical instantiation. Furthermore, we show that the components of the algorithm can be implemented in extremely noisy, low-power, sub-threshold logic.

Introduction

• Control algorithms need to deal with changing plants
  • Sudden changes like dealing with changes in weight, air friction (wind)
  • Gradual changes like changing motor characteristics, or length of limbs (for people, anyway)
• Not worrying about optimal control
• Need something that will work, and work quickly
• We also want something that is always on and always adapting, but doesn't cause excessive power draw
  • Data from quadcopters on how much power is used for control versus flight

Adaptive Tracking Control

• Dynamics:
  • $ s = {K_p} (q_d-q) + K_v {d \over {dt}} (q_d-q) $
  • $\tau = s + Y_d \theta_d $
  • $\Delta\theta_d = L_d Y_d^T s$
  • Learns an additive term onto the control signal that compensates

• Kinematics:

  • $s = {K_p} J^T (x_d-x) - K_v \dot{q}$
  • $ \dot{x} = J(q,\theta_k)\dot{q} = Y_k(q, \dot{q})\theta_k $
  • $ \Delta{\theta}_k = L_k Y_k^T K_p (x_d - x) $

• These work over some pre-defined basis spaces $\theta_k$ and $\theta_d$. To be completely general, we can use some large basis space like gaussian networks (Sanner & Slotine, 1992).

Neuromorphic control

• Neurons can be thought of as defining a basis space
  • this is the basic assumption behind the idea of population codes and the NEF
• Every group of neurons represents a value in some space
• The activity of an individual neuron is some function of the state being represented
  • This can be any function, as long as there is a large degree of heterogeneity across the neurons
  • For example, $a_i = G_i[\alpha_i e_i \cdot x + \beta]$ where $a$ is the activity, $G_i$ is the response function (such as $tanh$), $\alpha$ is a random scalar gain, $e_i$ is the "preferred" state for this neuron (the one for which it is most active), $x$ is the state beign represented, and $\beta$ is a random scalar bais.
  • This has been used to map to a wide variety of biological systems (e.g. Spaun)
  • Recovering the original value from this basis space can be done extremely robustly with just $\hat{x}=\sum_i(a_i d_i)$
  • Works well with extremely noisy response functions $G_i$ (including even huge nonlinearities like spiking)
• We can recast the adaptive tracking control this way
  • $Y_d$ and $Y_k$ can be the neural activities
  • $\theta_d$ and $\theta_k$ are connection weights out of those neurons
  • the learning rules adjust the connection weights
• To do this, we need to change $Y_d$ and $Y_k$ to be vectors rather than matrices (and $\theta_d$ and $\theta_k$ become matrices instead of vectors -- or possibly tensors)
  • and the learning rules become a bit different too

Experiments

• Simulation of 3-joint arm moving to a point
• Tracking (?)
  • note: do we need to also bring in all the further details needed to do tracking right? Or is that too much to add in to this paper?
• Shadow control (because I really like that example)
• control in a physical arm too
  • assuming we get the arm working soon -- in theory the parts we've been waiting for have just been shipped!
  • If the arm works well, maybe do all the examples with a physical arm and ignore simulation
  • Are there things it's easier to explore in simulation?

Neuromorphic hardware

• Energy costs of implementing this algorithm on typical hardware
  • lots of "neurons" in order to get a wide range of basis functions
  • CPU (using our OpenCL implementation)
• What about on GPUs?
  • (also using our OpenCL implementation)
• But maybe efficient on modern massively parallel energy efficient hardware
  • SpiNNaker
  • Many ARM cores with parallel communication
• The whole point of the algorithm is that it works with large numbers of very noisy components
  • So let's do that
  • People working on custom chips that use extremely small transistor sizes
    • Transistor mismatch gives us the hetereogenity we need!
    • Subthreshold analog electronics
    • Stanford, Zurich, etc.
  • try this algorithm on Neurogrid
    • warning: the current version of Neurogrid doesn't support online changing of connection weights
    • but we should still be able to show the energy savings
• other options
  • Hassler's FPAAs
  • FPGAs?

Discussion

• What is this useful for?
• Always-on adaptive robotics applications
• Constantly adjusting control to get deal with changes to the environment/plant
• Compare to optimal control
  • Sure, it's not optimal, but optimal doesn't adjust to the new situation